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(57) Abstract 

A virtual retinal display utilizes photon generation and manipulation to create a panoramic, high resolution, color virtual 
image that is projected directly onto the retina of the eye without creating a real or an aerial image that is viewed via a mirror or 
optics (18). The virtual retinal display includes a source of photons (12), the photons being modulated (14) with video information 
(40) and scanned in a raster type of pattern directly onto the retina of the user's eye. The photon generator (12) may utilize coher- 
ent or non-coherent light. Further, the photon generator may utilize color light generators (80, 82, 84) so as to scan a colored virtu- 
al image directly onto the retina of the user's eye. The virtual retinal display may also include a depth accommodation cue to vary 
the focus of scanned photons rapidly so as to control the depth perceived by a user for each individual picture element of the vir- 
tual image. Further, an eye tracking system (106) may be utilized to sense the position of an entrance pupil of the user's eye, the 
detected pupil position being utilized to move the scanned photons so as to be approximately coincident with the entrance pupil 
of the eye. 
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VIRTUAL RETINAL DISPLAY 



TECHNICAL FIELD 
The present invention is directed to a virtual 
image display system and more particularly to a 
virtual retinal display wherein photons modulated 
5 with video information are projected directly onto 
the retina of the eye to produce a virtual image 
without a perceivable aerial image outside of the 
user's eye* 

BACKGROUND OF THE INVENTION 
With known virtual image displays, a user does 
not view directly a physical display screen such as 
with real image displays. Typically, the virtual 
display creates only a small physical image using a 
liquid crystal array, light emitting diodes or a 
miniature cathode ray tube, CRT, the image being 
projected by optical lenses and mirrors so that the 
image appears to be a large picture suspended in the 
world. 

A miniature cathode ray tube can produce a 
medium resolution monochrome picture. However, 
these devices are heavy and bulky. For example, a 
typical weight of a miniature CRT with cables is 
greater than four ounces, the CRT having a one inch 
diameter and a four inch length. Further, these 
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devices have high voltage acceleration potential, 
typically 7-13 kilovolts which is undesirably high 
for a display that is mounted on a user's head. 
Creating color using a single miniature CRT is 
5 difficult and usually causes significant compromises 
in image resolution and luminance. Although the CRT 
image may be relayed via a coherent fiber-optics 
bundle to allow the CRT to be located away from head 
mounted optics, the hardware to accomplish this is 

10 also heavy and causes significant light loss. Field 
sequential color using a multiplexed color filter 
and CRT with white phosphor is able to create good 
color hue saturation but also at a significantly 
reduced resolution. For example, three color fields 

15 must be produced during the same period as a normal 
60Hz field, thereby dividing the video bandwidth for 
each color by three. 

A liquid crystal array can produce a color 
image using a low operating voltage, but it can 

20 provide only a marginal picture element (pixel) 
density, i.e. less than 800 by 800 elements. One 
commercial device is known that uses a linear array 
of light emitting diodes viewed via a vibrating 
mirror and a simple magnifier. Although this is a 

2 5 low cost and low power alternative, the display is 

monochrome and limited in line resolution to the 
number of elements which can be incorporated into 
the linear array. 

Both the CRT and liquid crystal display 

3 0 generate real images which are relayed to the eyes 

through an infinity optical system. The simplest 
optical system allows a user to view the image 
source through a simple magnifier lens. For fields 
of view greater than 30°, this approach leads to a 
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number of problems including light loss and 
chromatic aberrations. Further, these optics are 
bulky and heavy. 

Virtual projection optical designs create an 
5 aerial image somewhere in the optical path at an 
image plane which is then viewed as an erect virtual 
image via an eye piece or objective lens. This 
approach increases the flexibility by which the 
image from the image source can be folded around the 

10 user's head for a head mounted display system, but 
large fields of view require large and bulky 
reflective and refractive optical elements. 

In addition to resolution limitations, current 
systems also have bandwidth deficiencies. Bandwidth 

15 is a measure of how fast the display system can 
address, modulate or change the light emissions of 
the display elements of the image source. The 
bandwidth of the display image source is computed on 
the basis of the number of elements which must be 

2 0 addressed over a given period of time. Addressing 
elements temporally is needed to refresh or maintain 
a perceived luminance of each element taking into 
account the light integration dynamics of retinal 
receptors and the rate at which information is 

2 5 likely to change. The minimum refresh rate is a 

function of the light adaptive state of the eye, 
display luminance, and pixel persistence, i.e. the 
length of time the picture element produces light 
after it has been addressed. Minimum refresh rates 

3 0 of 50 to 60 times a second are typically needed for 

television type displays. Further, an update rate 
of at least 3 0Hz is needed to perceive continuous 
movement in a dynamic display or in a presentation 
in which the display image is stabilized as a result 
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of head movement. Refreshing sequentially, i.e. one 
element at a time, 40 million picture elements at a 
60Mhz rate would require a video bandwidth of 
2.4GHz. Bandwidth requirements can be reduced by 
interlacing which tricks the eye in its perception 
of flicker but still requires that all of the 
elements of the image source be addressed to achieve 
a minimum update rate of 30Hz or 1.2GHz bandwidth. 
Typical television broadcast quality bandwidths are 
approximately 8MHz, or two orders of magnitude less 
than the 1.2GHz. High resolution computer terminals 
have 1400 by lioo picture elements which are 
addressed at a 70Hz non-interlaced rate which is the 
equivalent to a bandwidth of approximately 100MHz. 



SUMMARY OF THE INVENTION 
In accordance with the present invention, the 
disadvantages of prior virtual image display systems 
have been overcome. The virtual retinal display of 
the present invention utilizes photon generation and 
manipulation to create a panoramic, high resolution, 
color virtual image that is projected directly onto 
the retina of the eye. The entrance pupil of the 
eye and the exit pupil or aperture of the virtual 
retinal display are coupled so that modulated light 
from a photon generator is scanned directly on to 
the retina producing the perception of an erect 
virtual image without an image plane outside of the 
user's eye, there being no real or aerial image that 
is viewed via a mirror or optics. 

More particularly, the virtual retinal display 
system of the present invention includes a source of 
photons modulated with video information, the 
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photons being scanned directly onto the retina of 
the user's eye. The photon generator utilized may 
produce coherent light such as a laser or it may 
produce non-coherent light. Further, the photon 
5 generator may include colored light generators such 
as red, green and blue light emitting diodes or 
lasers to provide colored light that is modulated 
with respective RGB video information. If a blue 
light source is not available, a yellow light source 

10 such as a yellow light emitting diode or laser may 
be used. The video modulated colored photons are 
combined and then scanned onto the retina. 

The video modulated signals are preferably 
scanned in both a horizontal and a vertical 

15 direction so as to produce a modulated light raster 
that is projected directly onto the user's eye by 
projection optics. The projection optics may 
include a toroidal or spherical optical element such 
as a refractive lens, mirror, holographic element, 

20 etc. Further, this optical element may be a light 
occluding element or it may be light transmissive. 
A light transmissive optical element allows the 
virtual retinal display of the present invention to 
be a see through display wherein the displayed 

2 5 virtual image is perceived by the user to be 

superimposed on the real world. Further, the light 
transmissiveness of the optical element may be 
actively or passively variable. 

The virtual retinal display system of the 

3 0 present invention further includes a depth cue for 

3-D imaging so as to reduce problems of "simulator 
sickness" that may occur with known stereoscopic 
display systems. More particularly, the depth cue 
varies the focus, i.e. the convergence or divergence 
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of the scanned photons rapidly to control the depth 
perceived by the user for each picture element of 
the image. Depth information ihay be stored in a Z 
axis buffer or the like in a video memory in 
5 addition to the horizontal and vertical information 
typically stored in a video frame buffer. 

A pupil tracking system may be employed to move 
the position of the light raster projected onto the 
eye so that it approximately coincides with the 
10 entrance pupil of the user's eye. This feature 
increases the resolution of the virtual retinal 
display and further increases the field of view to 
provide a fully immersive environment such that as 
the eye moves to one side, a view corresponding to 
15 that direction may be presented. This is 

accomplished by utilizing the detected pupil 
position to position a "visible window" on the video 
information stored in the frame buffer. The frame 
buffer may for example store video information 

2 0 representing a panoramic view and the position of 

the visible window determines which part of the view 
the user is to perceive, the video information 
falling within the visible window being used to 
modulate the light from the photon generator. 
25 The virtual display system of the present 

invention may also divide the video information into 
sectors or regions and use parallel photon 
generation and modulation to obtain ideal pixel 
density resolution across very wide fields of view. 

3 0 Further, by allowing the overall pixel density to be 

divided into separately scanned regions the 
bandwidth is reduced by the number of regions so as 
to overcome the bandwidth problems of prior systems. 
The virtual retinal display of the present 
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invention produces a wide field of view presentation 
which can reach 140° horizontally depending upon the 
size and proximity of the final optical element 
relative to the entrance pupil of the eye* Further , 
5 the virtual retinal display of the present invention 
is very small in size, weight and bulk since it is 
not necessary to produce either or a real or an 
aerial image. Because of its small size, weight and 
compactness the virtual retinal display is ideally 

10 suited for mounting on a user's head. 

These and other objects, advantages and novel 
features of the present invention as well as details 
of an illustrated embodiment thereof, will be more 
fully understood from the following description and 

15 the drawing. 

BRIEF DESCRIPTION OF THE DRAWING 
Fig. 1 is a block diagram of the virtual 
retinal display of the present invention; 

Fig. 2 is a block diagram illustrating one 
20 embodiment of the virtual retinal display depicted 
in Fig. 1; 

Fig. 3 is a second embodiment of the virtual 
retinal display of Fig. 1 utilizing color; 

Fig. 4 is a block diagram illustrating another 
25 embodiment of a color virtual retinal display in 
accordance with the present invention; 

Fig. 5 is a diagram of an LED array utilized in 
a further embodiment of the virtual retinal display 
of the present invention employing parallel photon 
3 0 generation and modulation; 

Fig. 6 is an illustration of a laser phased 
array ; 

Fig. 7 is an illustration of a microscanner 
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utilized in accordance with the present invention; 
and 

Fig. 8 is an illustration of another 
microscanner that may be utilized in accordance with 
the present invention. 



DESCRIPT ION OF THE PREFERRED EMBODIMENTS 
The virtual retinal display 10 of the present 
invention as shown in Fig. 1 utilizes photon 
generation and manipulation capable of creating a 
panoramic, high resolution, color image that is 
pro j ected directly onto the eye of a user , there 
being no aerial image or image plane outside of the 
eye that is viewed by the user. The virtual retinal 
display does not use a display that generates a real 
image such as a CRT, LCD or LED array as in prior 
virtual image displays. Nor does the virtual 
retinal display 10 need the mirrors or optics 
necessary in prior virtual image displays to 
generate an aerial image. Instead, photons 

modulated with video information are scanned 
directly onto the retina 22 of a user's eye 20 to 
produce the perception of an erect virtual image. 
Because the virtual retinal display 10 does not 
utilize a real image display or the mirrors or 
optics necessary to generate an aerial image, the 
virtual retinal display 10 is small in size and 
weight and is therefore suitable to be easily 
mounted on the user's head as a head mounted 
display. 

More particularly, as shown in Fig. l, photons 
from a photon generator 12 are modulated with video 
information by a modulator 14 . The modulated 
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photons are scanned in a first direction and in a 
second direction generally perpendicular to the 
first direction by a scanner 16 to create a raster 
of photons that is projected directly onto the 
5 retina 22 of the eye 20 of the user by projection 
optics 18 to produce the perception of an erect 
virtual image without an aerial image or image plane 
outside of the eye that is viewed or perceived by 
the user. Although not necessary, it is desirable 
10 to employ an eye tracking system 2 4 to reposition 
the scanned raster of light as the pupil 2 6 of the 
eye 2 0 moves so that the light ray bundles are 
coincident with the entrance pupil of the eye. The 
eye tracking system 24 can also be used as feedback 
15 to change the image or the focus of the image 
scanned onto the retina as the eye moves so that the 
user perceives that he is focusing on a different 
portion of a panoramic scene as he shifts his eye. 
It is noted that the dotted lines shown entering the 
20 eye 2 0 in Fig. 1 as well as in subsequent figures 
represents the range of scanning and not the 
instantaneous ray bundle. 

The photon generator 12 may generate coherent 
light such as a laser or it may generate non- 
25 coherent light such as by utilizing one or more 
LEDs. Further, beams of red, green and yellow or 
blue light may be modulated by RGY or RGB video 
signals to scan colored photons directly onto the 
user's eye. In order to reduce the bandwidth of the 
30 virtual retinal display, multiple monochromatic 
beams or multiple groups of colored beams can be 
modulated and scanned in parallel onto the retina 
where the video information used to modulate the 
photons is divided into different sectors or regions 
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and each beam or group of colored beams is 
associated with a different sector of video 
information as described below. It is further noted 
that the functions performed by one or more of the 
5 photon generator 12, modulator 14, scanner 16 and 
projection optics 18 can be combined to be performed 
by fewer elements depending upon the actual 
components used in the system. For example, an 
acousto-optic deflector may be used to both modulate 

10 the light from the photon generator 12 and to scan 
the modulated light in at least one direction. 
Further, a laser phased array may be utilized to 
perform the functions of the photon generator, 
modulator and one or possibly two scanners as 

15 discussed below. 

The components of the virtual retinal display 
10 can be made small, compact and lightweight so 
that the virtual retinal display JO can easily be 
mounted on the head of a user without requiring a 

2 0 helmet or an elaborate head mounting for structural 

support. Further, the photon generator 12 and 
modulator 14 can be separated from the scanner 16 
and projection optics 18 so that only the scanner 16 
and optics 18 need be mounted on the head of a user, 
25 the modulated photons being coupled to the scanner 
via one or more monofilament optical fibers. In a 
preferred embodiment, microscanners are utilized to 
scan the photons, such microscanners being small, 
thin and deflected to scan the photons in response 

3 0 to an electrical drive or deflection signal. The 

photon generator, modulator and scanner can 
therefore be made very small such as 1% inch high by 
1% inch wide by h inch thick or less with a weight 
of less than an ounce so as to facilitate a head 



BNSDOCID: <WO 94094 72A1_I_> 



WO 94/09472 




PCT/US93/09467 



- 11 - 



mounting for the virtual retinal display 10, 

In accordance with one embodiment of the 
present invention as shown in Fig. 2, high 
resolution scanners are used to deflect a beam of 
5 light both horizontally and vertically in a two 
dimensional raster pattern. No lens is used to 
focus the beam to form a real image in front of the 
eye. Instead, the lens 2 9 of the eye focuses the 
beam to a point on the back of the retina, the 

10 position of the beam point scanning the retina as 
the scanner 16 scans the modulated photons. The 
angle of deflection of the collimated light beams 
corresponds to the position of the focused spot on 
the retina for any given eye position just as if an 

15 image were scanned at an infinite distance away from 
the viewer. The intensity of the light is modulated 
by the video signal in order to create an image of 
desired contrast. Therefore, when the user's eye 
moves, the user will perceive a stationary image 

20 while he looks at different parts of the scene. The 
lateral extent of the image is proportional to the 
angle of the scan. Anamorphic optics are used as 
necessary to align the scanned photons and to scale 
the perceived image. By forming a reduced image of 

25 the scanner aperture, a proportionately larger 
scanning angle is yielded. Other than this, the 
size of the scanner image is irrelevant as long as 
the light enters the eye. 

More particularly, as shown in Fig. 2, light or 

30 photons from a photon generator 12 is projected 
through a cylindrical lens 3 0 and a spherical lens 
32 to an acousto-optical deflector 34 that scans the 
photons in a first or horizontal direction. The 
cylindrical lens spreads the light beam from the 
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photon generator 12 horizontally so that it fills 
the aperture of the acousto-optical deflector 34. 
The spherical lens 3 2 horizontally collimates the 
light which impinges onto the acousto-optical 
5 deflector 34. 

The acousto-optical deflector 34 is responsive 
to a video signal on a line 3 6 that is applied as a 
drive signal to a transducer of the acousto-optic 
deflector 34 to modulate the intensity of the 

10 photons or light from the photon generator 12 and to 
scan the modulated light from the photon generator 
12 in a first direction or horizontally. The video 
signal on line 3 6 is provided by a video drive 
system generally designated 38 that includes a video 

15 controller 42. The video controller 42 may include 
a video generator such as a frame buffer 4 0 that 
provides video signals on a line 56 and respective 
horizontal sync and vertical sync signals. The 
video controller 42 may also include a 

20 microprocessor that operates in accordance with 
software stored in a ROM 4 6 or the like and utilizes 
a RAM 48 for scratch pad memory. The horizontal 
sync signal from the video generator 4 0 is converted 
to a ramp wave form by a ramp generator 50, the 

25 horizontal sync ramp waveform is applied to a 
voltage controlled oscillator 52 that provides a 
signal in response to the ramp input having a 
frequency that varies such that it chirps. The 
output from the voltage controlled oscillator 52 is 

30 applied to an amplifier 54 the gain of which is 
varied by the video data signal 56 output from the 
video generator 4 0 so that the video signal 3 6 
output from the amplifier 54 has an amplitude that 
varies in accordance with the video information on 
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line 56 and that has a frequency that varies in a 
chirped manner. The video signal on line 3 6 is 
applied to a drive transducer of the acousto-optical 
deflector 34. Varying the amplitude of the drive 
5 signal on line 3 6 with the video information causes 
the acousto-optical deflector 34 to modulate the 
intensity of the light from the photon generator 12 
with the video information. Varying the frequency 
of the drive signal on line 3 6 in a chirped manner 
10 causes the acousto-optical deflector to vary the 
angle at which the light is deflected thereby so as 
to scan the light in a first or horizontal 
direction. 

A spherical lens pair 64 and 68 images the 

15 horizontally scanned light or photons onto a 
vertical scanner 62 wherein a cylindrical lens 68 
spreads the light vertically to fill the aperture of 
the vertical scanner 62. The vertical scanner 62 
may for example be a galvanometer. The vertical 

20 sync signal output from the video generator 4 0 is 
converted to a ramp waveform by a ramp generator 58 
and amplified by an amplifier 60 to drive the 
vertical scanner 62 . The speed of scanning of the 
vertical scanner 62 is slower than the scanning of 

25 the horizontal scanner 3 4 so that the output of the 
vertical scanner 62 is a raster of photons. This 
raster of photons is projected directly onto the eye 
20 of the user by projection optics taking the form 
of a toroidal or spherical optical element 72 such 

30 as a refractive lens, mirror, holographic element, 
etc. 

The toroidal or spherical optical element 72 
provides the final imaging and reduction of the 
scanned photons. More particularly, the toroidal or 
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spherical optical element relays the scanned photons 
so that they are coincident near the entrance pupil 
2 6 of the eye 20. Because a reduced image of the 
scanner aperture is formed, the deflection angles 
are multiplied in accordance with the Lagrange 
invariant wherein the field of view and image size 
are inversely proportional. As the size of the 
scanned photons, i.e. the exit aperture of the 
virtual retinal display are reduced, the field of 
view of the image perceived by the eye increases. 

The optical element 72 can be an occluding 
element that does not transmit light from outside of 
the display system. Alternatively the optical 
element 72 can be made light transmissive to allow 
the user to view the real world through the element 
72 wherein the user perceives the scanned virtual 
image generated by the display 10 superimposed on 
the real world. Further, the optical element 72 can 
be made variably transmissive to maintain the 
contrast between the outside world and the displayed 
virtual image. A passively variable light 

transmissive element 72 may be formed by sandwiching 
therein a photochromic material that is sensitive to 
light to change the light transmissiveness of the 
element as a function of the ambient light . An 
actively variable light transmissive element 72 may 
include a liquid crystal material. A photosensor 
can be used with such an element to detect the 
amount of ambient light wherein a bias voltage 
across the liquid crystal material is varied in 
accordance with the detected light to actively vary 
the light transmissiveness of the element 72. 

The system described thus far with respect to 
Fig. 2 is monocular. In order to provide a 
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stereoscopic system a second virtual retinal display 
10' may be utilized in parallel with the first 
retinal display 10, the second virtual retinal 
display 10* projecting scanned photons modulated 
5 with the appropriate video information directly on 
the second eye 20* of the user* This provides a 
medium for binocular depth information so that 
displayed objects appear at different depths. Each 
pixel of the object, however, appears at the same 
10 distance from the user which can create a possible 
conflict between the stereoscopic cue and the 
monocular cue where the stereoscopic cue deals with 
the positioning of the object with respect to each 
eye and the monocular cue deals with the focus of 
15 the light of the object being imaged on the retina. 

More particularly, in prior virtual image display 
systems, each monocular image plane was typically 
focused at optical infinity causing each of the 
pixels within the virtual image to appear at one 
2 0 distance. However, the combination of two prior 
monocular systems to form the binocular view created 
a possible conflict between the distance cues and 
the focus or accommodation cue. 

The virtual retinal display of the present 
25 invention overcomes this problem by utilizing an 
accommodation cue 70 either in the monocular display 
system 10 or in the binocular display system formed 
of displays 10 and 10 1 . The accommodation cue 70 is 
a focusing or depth cue that is controlled to vary 
30 the focus or convergence or divergence of the 
scanned photons rapidly to control the depth 
perceived for each picture element of the virtual 
image. Therefore in accordance with the present 
invention true depth perception is obtained by 
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modulating each pixel for depth individually such as 
by controlling the focus, i.e. the convergence or 
divergence, of the individual pixel. The 
accommodation cue 70 includes a reflective surface 
5 that changes shape rapidly. For example, a 

miniature mirror having a deformable membrane whose 
shape is altered as the membrane is charged and 
discharged may be used to form the accommodation 
cue. The deformation of the membrane is thus varied 

10 by an electrical drive signal to control the 
convergence or divergence of each pixel for depth. 
The drive of the accommodation cue 7 0 is provided by 
the video controller 42 which may, for example, 
store a Z axis video information buffer in the 

15 memory 48 or in the video generator 40 in addition 
to the two dimensional video information in a 
typical frame buffer. 

A further embodiment of the virtual retinal 
display 10 of the present invention is depicted in 

2 0 Fig. 3 for scanning colored photons directly onto 
the retina of a user's eye. As shown in Fig. 3, the 
photon generator 12 includes colored lasers or LEDs 
such as a red photon generator 80, a green photon 
generator 82 and a blue photon generator 84. If a 

2 5 blue photon generator is unavailable, a yellow 
photon generator may be utilized. The colored 
photons from the generators 80, 82 and 84 are 
modulated with respective RGB video information from 
the video generator 40 and then combined by a beam 

30 combiner/dispersion precompensator 86. The output 
of the beam combiner/dispersion precompensator 86 is 
projected onto the horizontal scanner 34 by the 
cylindrical lens 30 and the spherical lens 32. It 
is noted tht the horizontal scanner may be other 
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than the acousto-optic scanner shown in Fig. 2. For 
example, a resonant mechanical scanner or various 
types of microscanners as discussed below may be 
used for the horizontal scanner. The horizontally 
5 scanned color modulated photons output from the 
scanner 34 are projected onto a dispersion 
compensator 88 the output of which is projected onto 
a prism before being projected onto the vertical 
scanner 62 by the spherical lens pair 64 and 68, 

10 The colored photon raster as scanned from the 

output of the vertical scanner 62 is projected by a 
spherical lens 92 onto an offset mirror 96 which is 
moved by the eye tracker 106 so as to position the 
raster of photons directly onto the entrance pupil 

15 26 of the eye 20 as the pupil moves. In one 
embodiment, a beam splitter 100 directs an image 
reflected off of the cornea of the eye 20 to a lens 
102 and a position sensing diode 104 that is coupled 
to the eye tracker 106 to detect the position of the 

20 pupil 26. In response to the detected position of 
the pupil, the eye tracker correctly positions the 
offset mirror (s) 96 so that the exit pupil or 
aperture of the virtual retinal display is 
approximately aligned with the entrance pupil of the 

25 eye and/or to adjust the scan angle to reflect 
changed video information as described below. 

The instantaneous position of the pupil 26 as 
determined by the eye tracker 106 is also 
communicated to the video controller 42 so that the 

3 0 microprocessor 44 can direct video information to 
modulate the colored light where the video 
information reflects a change in the direction of 
the user's view. More particularly, the detected 
pupil position is used by the microprocessor 44 to 
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position a "visible window" on the video information 
stored in the frame buffer 40. The frame buffer 40 
may for example store video information representing 
a panoramic view and the position of the visible 
5 window determines which part of the view the user is 
to perceive, the video information falling within 
the visible window being used to modulate the light 
from the photon generator 12, 

It is noted that, because the acousto-optical 

10 deflector 3 4 diffracts red light more than green 
light and diffracts green light more than blue 
light, this variation in the diffraction must be 
compensated for. In accordance with the present 
invention, this variation in diffraction may be 

15 compensated for by appropriately delaying via delays 
108, 110 and 112 the RGB video signals that are 
coupled to the respective red, green and blue photon 
generators 80, 82 and 8 4 to modulate the red, green 
and blue photons with the appropriate red, green and 

2 0 blue video information. 

In another embodiment of the virtual retinal 
display of the present invention as shown in Fig. 4, 
composite video or RGB video signals are received by 
a digital video scan converter 120 and separated 
25 into multiple compartments that represent sectors or 
regions of an image to be scanned. Multiple video 
drive signals output from the video amplifiers 124 
representing each sector are used to modulate the 
light from the photon generator 12 in parallel. The 

3 0 photon generator may consist of either arrays of 

lasing diodes or arrays of high luminance light 
emitting diodes. Multiple beams of red, green and 
yellow or blue light are modulated with the video 
signals in parallel for each of the divided sectors 
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or regions and then relayed directly or by 
monofilament optical fibers 131 to a microscanner 
16. The microscanner 16 essentially performs two 
functions. First, the microscanner scans the 
5 multiple color beams associated with each sector or 
region in two axes to create a raster of light on 
the retina and not an aerial image, there being no 
image plane between the photon generator 12 and the 
eye 20. Second, the microscanner 16 functions to 

10 position the scanned light relative to the 
instantaneous entrance pupil 2 6 of the eye as sensed 
by the eye tracker 24. 

More particularly, the scanner 16 includes a 
first microscanner 132 that is responsive to an X 

15 axis deflection signal output from a deflection 
amplifier 13 6 to scan the color beams in a 
horizontal direction where the amplifier 136 is 
driven by the horizontal sync sigral from a scan 
generator 122. A second microscanner 134 is 

2 0 responsive to a Y deflection signal from the 
deflection amplifiers 136 as driven by the vertical 
sync or deflection drive from the scan generator 122 
to scan the horizontally scanned color photons in 
the vertical direction. A scan collimation lens 140 

2 5 receives a two dimensional ly modulated light field 

that is projected onto a tri-color combiner 142. 
The combiner 142 in turn projects the scanned light 
onto a Maxwellian-view optical system 148. The 
optical system 148 projects the scanned colored 

3 0 photons onto a raster position deflector which may 

include two axis galvo mirrors that in turn project 
the scanned light onto a toroidal optical element 
such as a combiner 152 having a tr ichor ic coating, 
the toroidal combiner 152 projecting the scanned 
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color photons directly onto the eye 20. 

For eye tracking, the eye tracker 24 includes 
an infrared light source which illuminates the 
surface of the eye with low intensity infrared light 
either directly or indirectly as shown. The surface 
of the eye is viewed through the raster position 
deflector 150 via the combiner 142, a lens 140 and a 
charge coupled device, CCD, array 14 6. The signals 
from the CCD sensor 14 6 are processed by a pupil 
position processor 154 to generate null signals, AH 
and AV, that are coupled to respective color 
deflection amplifiers 158 and to the raster 
positioning mirrors 150 so as to cause the scanned 
photons to follow the pupil of the user's eye 20. 

An example of a light emitting diode array 
suitable for use in the present invention is 
illustrated in Fig. 5. If an X-Y visual field is 
considered to be composed of an array of 2,000 x 
2,000 resolvable spots or pixels, the spots must be 
refreshed 50 times per second so as to have an 
information bandwidth of approximately 200MHz. High 
brightness LEDs typically have a power bandwidth 
curve that starts to roll off above 2MHz. This 
result is essentially an R-C product limitation 
related to the diffusion capacitance of a heavily 
forward-biased p-n junction. In order to meet the 
bandwidth requirements of the system, a linear array 
of 50 to 100 LED pixels per color are utilized. 
Using a red, green and blue LED scheme would require 
50-100 LEDs of each of these three colors. As shown 
in Fig. 5, an array 200 includes LED chips 201, 202, 
2 03 - N wherein each LED chip includes an LED active 
area 205. The LED active area may include a GaAsP 
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A laser phased array as illustrated in Fig. 6 
functions to perform photon generation, video 
modulation and scanning in at least one direction. 
5 The laser phased array includes a thin film wave 
guide 210, phase modulator electrodes 212, a cleaned 
coupled cavity 214 and laser cavities 216, the array 
emitting a coherent beam of about lOmW power. 
When two closely spaced lasers are fabricated 
10 in the same chip of material, their optical fields 
become coupled so that the processes of optical 
emission in the two devices are correlated and 
coherent. The result is a well defined phase front 
emitted from the laser pair. In the laser phased 
15 array 22 0 having a number of laser cavities 216, the 
optical beam is phase coherent if the lasers are 
spaced within 10 microns of each other. This 
resolution can be achieved by photolithographic 
techniques. The electro-optic modulator works by 
20 modifying the index of refraction of the wave guide 
medium 210 through which the optical beam must 
travel before being launched into free space. By 
separating the electrical contacts 212 for each 
modulator, the relative phase of each individual 
25 laser in the array can be modified by the modulator. 
For an appropriate series of modulation voltages, 
the phase front of the laser array coupled beam can 
be modified so that the emitted beam is launched at 
an angle to the normal exit direction. With the 
30 appropriate series of modulation voltages the laser 
beam can be scanned in a given direction. It is 
possible to construct a two axis laser phased array 
so that an additional scanner is not needed to scan 
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the laser in a perpendicular direction. 

An example of a microscanner 132, 134 for 
scanning photons is illustrated in Fig. 7. The 
microscanner includes an actuator 23 0. The actuator 
5 23 0 is a piezoelectric bimorph cantilever that is 
capable of three dimensional motion in response to 
an electrical drive signal. By controlling the 
deflection of the cantilevered actuator with the 
appropriate drive signals, the actuator 23 0 deflects 

10 the photons incident thereto to scan the photons. 

Another example of a microscanner that can be 
.made extremely small is shown in Fig. 8, the 
microscanner having a curved reflective surface that 
translates to scan light impinging thereon in one 

15 direction. More particularly, the microscanner 24 0 
includes a base or actuator 242 formed of a 
piezoelectric material with a substrate 24 4 formed 
on the actuator 242 wherein the substrate 244 has a 
curved reflective surface 24 6. In response to a 

20 varying drive signal the piezoelectric actuator and 
the substrate 244 translate in the direction of the 
arrows 248 so as to scan the light impinging on the 
surface 24 6 of the substrate in a first direction 
generally perpendicular to the direction 248 of 

25 translation. A second microscanner 250 scans the 
light impinging thereon in a second direction 
perpendicular to the first direction so as to scan a 
raster image directly onto the retina of a user's 
eye. 

3 0 Many modifications and variations of the 

present invention are possible in light of the above 
teachings. Thus, it is to be understood that, 
within the scope of the appended claims, the 
invention may be practiced otherwise than as 
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described hereinabove. 
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CLAIMS 

1. A virtual image display system comprising: 
a source of photons modulated with video 

information, said photons being scanned directly 
onto the retina of the user's eye to produce a 
virtual image perceived by the user without an 
aerial image outside of the user's eye perceivable 
by the user . 

2. A virtual image display system as recited 
in claim 1 wherein said display system is mounted on 
a user's head. 

3. A virtual image display system as recited 
in claim 1 wherein said source of photons generates 
coherent 1 ight . 

4. A virtual image display system as recited 
in claim 1 wherein said source of photons includes 
colored light generators. 

5. A virtual image display system as recited 
in claim 1 wherein said source of photons includes 
an array of non-coherent light emitting elements. 

6. A virtual image display system as recited 
in claim 1 wherein said source of photons includes a 
laser phased array. 



WO 94/09472 




PCT/US93/09467 



- 25 - 



7. A virtual image display system as recited 
in claim 1 wherein said video information represents 
a plurality of picture elements of a video image and 
said system includes means for varying the focus of 

5 said scanned photons to control the depth perceived 
for each picture element of said video image. 

8. A virtual image display system as recited 
in claim 1 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a photon deflector for moving said 

5 scanned light to approximately coincide with said 
entrance pupil. 

9 . A virtual image display system as recited 
in claim 8 including a source of video information, 
said video information source being responsive to 
movement of said pupil position to vary the video 

5 information with which said photons are modulated. 

10. A virtual image display system as recited 
in claim 1 wherein said video information includes a 
plurality of video signals representing a plurality 
of regions of a video image , said photons being 

5 modulated by said plurality of video signals in 
parallel and scanned in parallel. 
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11. A virtual image display system to be 
mounted on the head of a user comprising: 

a source of photons modulated with video 
information, said photons being scanned in a first 
5 direction and in a second direction generally 
perpendicular to said first direction directly onto 
the retina of the user's eye to produce a virtual 
image perceived by the user without an aerial image 
outside of the user's eye perceivable by the user. 

12 . A virtual image display system as recited 
in claim 11 wherein said source of photons generates 
coherent 1 ight . 

13 . A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
colored light generators. 

14. A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
an array of non-coherent light emitting elements. 

15. A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
a laser phased array. 

16. A virtual image display system as recited 
in claim 11 wherein said video information 
represents a plurality of picture elements of a 
video image and said system includes means for 

5 varying the focus of said scanned photons to control 
the depth perceived for each picture element of said 
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video image. 

17. A virtual image display system as recited 
in claim 11 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a photon deflector for moving said 

5 scanned light to approximately coincide with said 
entrance pupil. 

18. A virtual image display system as recited 
in claim 17 including a source of video information, 
said video information source being responsive to 
movement of said pupil position to vary the video 

5 information with which said photons are modulated. 

19. A virtual image display system as recited 
in claim 11 wherein said video information includes 
a plurality of video signals representing a 
plurality of regions of a video image, said photons 

5 being modulated by said plurality of video signals 
in parallel and scanned in parallel. 

20. A virtual image display system to be 
mounted on the head of a user comprising: 

a source of photons modulated with video 
information, said photons being scanned in a first 
5 direction and in a second direction generally 
perpendicular to said first direction; and 

optics to project said scanned photons directly 
onto the retina of the user • s eye to produce a 
virtual image perceived by the user without an 
10 aerial image outside of the user's eye perceivable 
by the user. 
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21. A virtual image display system as recited 
in claim 2 0 wherein said optics include a toroidal 
optical element . 

22. A virtual image display system as recited 
in claim 20 wherein said optics include a light 
transmissive optical element, said virtual retinal 
display forming a see-through display wherein said 

5 scanned photons are perceived by the user as being 
superimposed upon the real world. 

23. A virtual image display system as recited 
in claim 22 wherein said optical element has 
variable light transmissivity . 

24. A virtual image display system for 
projecting an image directly onto an eye of a user 
comprising: 

a source of light ; 
5 means for modulating light form said light 

source with a video signal; 

means for scanning said modulated light; and 
optics for projecting said scanned light 
directly onto an eye of the user to produce a 
10 virtual image perceived by the user without an 
aerial image outside of the user's eye perceivable 
by the user. 

25. A virtual image display system as recited 
in claim 24 wherein said display system is mounted 
on a user's head. 
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26. A virtual image display system as recited 
in claim 24 wherein said source of light generates 
coherent light. 

27. A virtual image display system as recited 
in claim 24 wherein said source of light includes 
colored light generators. 

28. A virtual image display system as recited 
in claim 2 4 wherein said source of light includes an 
array of non-coherent light emitting elements. 

29. A virtual image display system as recited 
in claim 24 wherein said source of light includes a 
laser phased array. 

30. A virtual image display system as recited 
in claim 24 wherein said video information 
represents a plurality of picture elements of a 
video image and said system includes means for 

5 varying the focus of said scanned light to control 
the depth perceived for each picture element of said 
video image. 

31. A virtual image display system as recited 
in claim 24 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a light deflector for moving said 

5 scanned light to approximately coincide with said 
entrance pupil. 
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32. A virtual image display system as recited 
in claim 31 including a source of video information, 
said video information source being responsive to 
movement of said pupil position to vary the video 

5 information with which said light is modulated. 

33. A virtual image display system as recited 
in claim 24 wherein said video information includes 
a plurality of video signals representing a 
plurality of regions of a video image, said light 

5 being modulated by said plurality of video signals 
in parallel and scanned in parallel. 

34. A virtual image display system as recited 
in claim 2 4 wherein said optics include a toroidal 
mirror. 

35. A virtual image display system as recited 
in claim 24 wherein said scanning means and optics 
are mounted on a head of a user , said source of 
light and modulating means are remote from said 

5 scanning means and including means for coupling said 
modulated light to said scanning means. 

36. A virtual image display system for 
projecting an image directly onto an eye of a user 
comprising: 

a source of light; 
5 a source of video information, said video 

information modulating light from said light source; 
means for scanning said video modulated light; 

and 

optics for projecting said scanned light 
10 directly onto a user's eye to produce a virtual 
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image perceived by the user without an aerial image 
outside of the user's eye perceivable by the user. 

37. A virtual image display system for 
projecting an image directly onto an eye of a user 
as recited in claim 3 6 wherein said light source is 
a source of coherent light. 

38 A virtual image display system for 
projecting an image directly onto an eye of a user 
as recited in claim 3 6 wherein said source of light 
is a source of non-coherent light. 

39. A display apparatus comprising: 
a light source for emitting a laser beam; 
an optical modulating system for optically 

modulating the laser beam from the light source in 
response to a video signal; 

a scanning system for horizontally and 
vertically scanning the modulated laser beam; and 

a projecting optical system for projecting the 
scanned laser beam from the scanned system to form 
an image of the laser beam on the retina of an eye 
of an observer of the display apparatus. 

40. A direct viewing picture image display 
apparatus comprising : 

a light source for emitting a laser beam; 
video signal supply means for supplying a video 
5 signal; 

modulating means for optically modulating the 
laser beam emitted from the light source in response 
to the video signal supplied from the video signal 
supplying means; 



5 



10 
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10 scanning signal supplying means for supplying a 

scanning signal; 

scanning means for horizontally and vertically 
scanning the modulated laser beam from the scanning 
signal in response to the scanning signal supplied 

15 from the scanning signal supplying means; and 

a projecting optical system for projecting the 
scanned laser beam received from the scanning means 
to form an image of the laser beam on the retina of 
an eye of an observer of the direct viewing picture 

2 0 image display apparatus. 

41. A direct viewing picture image display 
apparatus comprising: 

a light source for emitting a laser beam; 

video signal supplying means for supplying a 
5 video signal ; 

modulating means for optically modulating the 
laser beam emitted from the light source in response 
to the video signal supplied from the video signal 
supplying means; 
10 depth signal supplying means for supplying 

depth information ; 

a variable focus optical system for providing 
the depth information supplied from the depth signal 
supplying means to the modulated laser beam received 
15 from the modulating means; 

scanning signal supplying means for supplying a 
scanning signal; 

scanning means for scanning the laser beam 
received from the variable focus optical system in 
20 response to the scanning signal supplied from the 
scanning signal supplying means; and 

a projecting optical system for projecting the 
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scanned laser beam received from the scanning means 
to form an image of the laser beam on the retina of 
25 an eye of an observer of the direct viewing picture 
image display apparatus. 

42. The direct viewing picture image display 
apparatus as recited in claim 39 further including a 
variable focus optical system. 

43. An apparatus for directly displaying an 
image directly on the retina of an eye comprising: 

a plurality of colored lights; 

means for modulating each of said colored 
5 lights with video information; 

means for combining said modulated colored 
lights into a single beam; 

means £or horizontally and vertically scanning 
said single beam of light; and 
10 optics for projecting said scanned light onto 

an eye of a user. 
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VIRTUAL RETINAL DISPLAY 



TECHNICAL FIELD 
The present invention is directed to a virtual 
image display system and more particularly to a 
virtual retinal display wherein photons modulated 
5 with video information are projected directly onto 
the retina of the eye to produce a virtual image 
without a perceivable aerial image outside of the 
user's eye. 

BACKGROUND OF THE INVENTION 
10 With known virtual image displays , a user does 

not view directly a physical display screen such as 
with real image displays. Typically, the virtual 
display creates only a small physical image using a 
liquid crystal array, light emitting diodes or a 
15 miniature cathode ray tube, CRT, the image being 
projected by optical lenses and mirrors so that the 
image appears to be a large picture suspended in the 
world. 

A miniature cathode ray tube can produce a 
2 0 medium resolution monochrome picture. However, 
these devices are heavy and bulky. For example, a 
typical weight of a miniature CRT with cables is 
greater than four ounces, the CRT having a one inch 
diameter and a four inch length. Further, these 
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devices have high voltage acceleration potential, 
typically 7-13 kilovolts which is undesirably high 
for a display that is mounted on a user's head. 
Creating color using a single miniature CRT is 
5 difficult and usually causes significant compromises 
in image resolution and luminance. Although the CRT 
image may be relayed via a coherent fiber-optics 
bundle to allow the CRT to be located away from head 
mounted optics, the hardware to accomplish this is 

10 also heavy and causes significant light loss. Field 
sequential color using a multiplexed color filter 
and CRT with white phosphor is able to create good 
color hue saturation but also at a significantly 
reduced resolution* For example, three color fields 

15 must be produced during the same period as a normal 
60Hz field, thereby dividing the video bandwidth for 
each color by three. 

A liquid crystal array can produce a color 
image using a low operating voltage, but it can 

2 0 provide only a marginal picture element (pixel) 
density, i.e. less than 800 by 800 elements. One 
commercial device is known that uses a linear array 
of light emitting diodes viewed via a vibrating 
mirror and a simple magnifier. Although this is a 

2 5 low cost and low power alternative, the display is 

monochrome and limited in line resolution to the 
number of elements which can be incorporated into 
the linear array. 

Both the CRT and liquid crystal display 

3 0 generate real images which are relayed to the eyes 

through an infinity optical system. The simplest 
optical system allows a user to view the image 
source through a simple magnifier lens. For fields 
of view greater than 30° , this approach leads to a 
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number of problems including light loss and 
chromatic aberrations. Further, these optics are 
bulky and heavy. 

Virtual projection optical designs create an 
5 aerial image somewhere in the optical path at an 
image plane which is then viewed as an erect virtual 
image via an eye piece or objective lens. This 
approach increases the flexibility by which the 
image from the image source can be folded around the 
10 user's head for a head mounted display system, but 
large fields of view require large and bulky 
reflective and refractive optical elements. 

In addition to resolution limitations, current 
systems also have bandwidth deficiencies. Bandwidth 
15 is a measure of how fast the display system can 
address, modulate or change the light emissions of 
the display elements of the image source. The 
bandwidth of the display image source is computed on 
the basis of the number of elements which must be 

2 0 addressed over a given period of time. Addressing 

elements temporally is needed to refresh or maintain 
a perceived luminance of each element taking into 
account the light integration dynamics of retinal 
receptors and the rate at which information is 
25 likely to change. The minimum refresh rate is a 
function of the light adaptive state of the eye, 
display luminance, and pixel persistence, i.e. the 
length of time the picture element produces light 
after it has been addressed. Minimum refresh rates 

3 0 of 50 to 60 times a second are typically needed for 

television type displays. Further, an update rate 
of at least 30Hz is needed to perceive continuous 
movement in a dynamic display or in a presentation 
in which the display image is stabilized as a result 
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of head movement. Refreshing sequentially, i.e. one 
element at a time, 40 million picture elements at a 
60Mhz rate would require a video bandwidth of 
2 . 4GHz . Bandwidth requirements can be reduced by 
5 interlacing which tricks the eye in its perception 
of flicker but still requires that all of the 
elements of the image source be addressed to achieve 
a minimum update rate of 3 0Hz or 1.2GHz bandwidth. 
Typical television broadcast quality bandwidths are 
10 approximately 8MHz, or two orders of magnitude less 
than the 1.2GHz. High resolution computer terminals 
have 1400 by 1100 picture elements which are 
addressed at a 70Hz non-interlaced rate which is the 
equivalent to a bandwidth of approximately 100MHz. 



15 SUMMARY OF THE INVENTION 

In accordance with the present invention, the 
disadvantages of prior virtual image display systems 
have been overcome. The virtual retinal display of 
the present invention utilizes photon generation and 

2 0 manipulation to create a panoramic, high resolution, 

color virtual image that is projected directly onto 
the retina of the eye. The entrance pupil of the 
eye and the exit pupil or aperture of the virtual 
retinal display are coupled so that modulated light 
25 from a photon generator is scanned directly on to 
the retina producing the perception of an erect 
virtual image without an image plane outside of the 
user's eye, there being no real or aerial image that 
is viewed via a mirror or optics. 

3 0 More particularly, the virtual retinal display 

system of the present invention includes a source of 
photons modulated with video information, the 
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photons being scanned directly onto the retina of 
the user's eye. The photon generator utilized may 
produce coherent light such as a laser or it may 
produce non-coherent light. Further, the photon 
5 generator may include colored light generators such 
as red, green and blue light emitting diodes or 
lasers to provide colored light that is modulated 
with respective RGB video information. If a blue 
light source is not available, a yellow light source 

10 such as a yellow light emitting diode or laser may 
be used. The video modulated colored photons are 
combined and then scanned onto the retina. 

The video modulated signals are preferably 
scanned in both a horizontal and a vertical 

15 direction so as to produce a modulated light raster 
that is projected directly onto the user's eye by 
projection optics. The projection optics may 
include a toroidal or spherical optical element such 
as a refractive lens, mirror, holographic element, 

2 0 etc. Further, this optical element may be a light 

occluding element or it may be light transmissive . 
A light transmissive optical element allows the 
virtual retinal display of the present invention to 
be a see through display wherein the displayed 
25 virtual image is perceived by the user to be 
superimposed on the real world. Further, the light 
transmissiveness of the optical element may be 
actively or passively variable. 

The virtual retinal display system of the 

3 0 present invention further includes a depth cue for 

3-D imaging so as to reduce problems of "simulator 
sickness" that may occur with known stereoscopic 
display systems. More particularly, the depth cue 
varies the focus, i.e. the convergence or divergence 
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of the scanned photons rapidly to control the depth 
perceived by the user for each picture element of 
the image. Depth information may be stored in a Z 
axis buf f er or the 1 ike in a video memory in 
addition to the horizontal and vertical information 
typically stored in a video frame buffer. 

A pupil tracking system may be employed to move 
the position of the light raster projected onto the 
eye so that it approximately coincides with the 
entrance pupil of the user • s eye . This feature 
increases the resolution of the virtual retinal 
display and further increases the field of view to 
provide a fully immersive environment such that as 
the eye moves to one side, a view corresponding to 
that direction may be presented. This is 

accomplished by utilizing the detected pupil 
position to position a "visible window" on the video 
information stored in the frame buffer. The frame 
buffer may for example store video information 
representing a panoramic view and the position of 
the visible window determines which part of the view 
the user is to perceive, the video information 
falling within the visible window being used to 
modulate the light from the photon generator. 

The virtual display system of the present 
invention may also divide the video information into 
sectors or regions and use parallel photon 
generation and modulation to obtain ideal pixel 
density resolution across very wide fields of view. 
Further, by allowing the overall pixel density to be 
divided into separately scanned regions the 
bandwidth is reduced by the number of regions so as 
to overcome the bandwidth problems of prior systems. 
The virtual retinal display of the present 
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invention produces a wide field of view presentation 
which can reach 140° horizontally depending upon the 
size and proximity of the final optical element 
relative to the entrance pupil of the eye. Further, 
5 the virtual retinal display of the present invention 
is very small in size, weight and bulk since it is 
not necessary to produce either or a real or an 
aerial image. Because of its small size, weight and 
compactness the virtual retinal display is ideally 

10 suited for mounting on a user's head. 

These and other objects, advantages and novel 
features of the present invention as well as details 
of an illustrated embodiment thereof, will be more 
fully understood from the following description and 

15 the drawing . 

BRIEF DESCRIPTION OF THE DRAWING 
Fig. 1 is a block diagram of the virtual 
retinal display of the present invention; 

Fig. 2 is a block diagram illustrating one 

2 0 embodiment of the virtual retinal display depicted 

in Fig. 1; 

Fig. 3 is a second embodiment of the virtual 
retinal display of Fig. 1 utilizing color; 

Fig. 4 is a block diagram illustrating another 
25 embodiment of a color virtual retinal display in 
accordance with the present invention; 

Fig. 5 is a diagram of an LED array utilized in 
a further embodiment of the virtual retinal display 
of the present invention employing parallel photon 

3 0 generation and modulation; 

Fig. 6 is an illustration of a laser phased 
array ; 

Fig. 7 is an illustration of a microscanner 
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utilized in accordance with the present invention; 
and 

Fig. 8 is an illustration of another 
microscanner that may be utilized in accordance with 
5 the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The virtual retinal display 10 of the present 
invention as shown in Fig. 1 utilizes photon 
generation and manipulation capable of creating a 

10 panoramic, high resolution, color image that is 
projected directly onto the eye of a user, there 
being no aerial image or image plane outside of the 
eye that is viewed by the user. The virtual retinal 
display does not use a display that generates a real 

15 image such as a CRT, LCD or LED array as in prior 
virtual image displays. Nor does the virtual 
retinal display 10 need the mirrors or optics 
necessary in prior virtual image displays to 
generate an aerial image- Instead, photons 

2 0 modulated with video information are scanned 
directly onto the retina 22 of a user's eye 20 to 
produce the perception of an erect virtual image. 
Because the virtual retinal display 10 does not 
utilize a real image display or the mirrors or 

2 5 optics necessary to generate an aerial image, the 

virtual retinal display 10 is small in size and 
weight and is therefore suitable to be easily 
mounted on the user ' s head as a head mounted 
display. 

3 0 More particularly, as shown in Fig. 1, photons 

from a photon generator 12 are modulated with video 
information by a modulator 14 . The modulated 
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photons are scanned in a first direction and in a 
second direction generally perpendicular to the 
first direction by a scanner 16 to create a raster 
of photons that is projected directly onto the 
5 retina 22 of the eye 20 of the user by projection 
optics 18 to produce the perception of an erect 
virtual image without an aerial image or image plane 
outside of the eye that is viewed or perceived by 
the user. Although not necessary, it is desirable 
10 to employ an eye tracking system 24 to reposition 
the scanned raster of light as the pupil 26 of the 
eye 2 0 moves so that the light ray bundles are 
coincident with the entrance pupil of the eye. The 
eye tracking system 24 can also be used as feedback 
15 to change the image or the focus of the image 
scanned onto the retina as the eye moves so that the 
user perceives that he is focusing on a different 
portion of a panoramic scene as he shifts his eye. 
It is noted that the dotted lines shown entering the 
20 eye 2 0 in Fig. 1 as well as in subsequent figures 
represents the range of scanning and not the 
instantaneous ray bundle. 

The photon generator 12 may generate coherent 
light such as a laser or it may generate non- 
25 coherent light such as by utilizing one or more 
LEDs. Further, beams of red, green and yellow or 
blue light may be modulated by RGY or RGB video 
signals to scan colored photons directly onto the 
user's eye. In order to reduce the bandwidth of the 
3 0 virtual retinal display, multiple monochromatic 
beams or multiple groups of colored beams can be 
modulated and scanned in parallel onto the retina 
where the video information used to modulate the 
photons is divided into different sectors or regions 
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and each beam or group of colored beams is 
associated with a different sector of video 
information as described below. It is further noted 
that the functions performed by one or more of the 
5 photon generator 12, modulator 14, scanner 16 and 
projection optics 18 can be combined to be performed 
by fewer elements depending upon the actual 
components used in the system . For example , an 
acousto-optic deflector may be used to both modulate 

10 the light from the photon generator 12 and to scan 
the modulated light in at least one direction. 
Further, a laser phased array may be utilized to 
perform the functions of the photon generator, 
modulator and one or possibly two scanners as 

15 discussed below* 

The components of the virtual retinal display 
10 can be made small, compact and lightweight so 
that the virtual retinal display 10 can easily be 
mounted on the head of a user without requiring a 

2 0 helmet or an elaborate head mounting for structural 
support. Further, the photon generator 12 and 
modulator 14 can be separated from the scanner 16 
and projection optics 18 so that only the scanner 16 
and optics 18 need be mounted on the head of a user, 

2 5 the modulated photons being coupled to the scanner 
via one or more monofilament optical fibers . In a 
preferred embodiment, microscanners are utilized to 
scan the photons, such microscanners being small, 
thin and deflected to scan the photons in response 

30 to an electrical drive or deflection signal. The 
photon generator, modulator and scanner can 
therefore be made very small such as 1% inch high by 
1% inch wide by h inch thick or less with a weight 
of less than an ounce so as to facilitate a head 
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mounting for the virtual retinal display 10. 

In accordance with one embodiment of the 
present invention as shown in Fig. 2, high 
resolution scanners are used to deflect a beam of 
5 light both horizontally and vertically in a two 
dimensional raster pattern. No lens is used to 
focus the beam to form a real image in front of the 
eye. Instead, the lens 29 of the eye focuses the 
beam to a point on the back of the retina, the 

10 position of the beam point scanning the retina as 
the scanner 16 scans the modulated photons. The 
angle of deflection of the collimated light beams 
corresponds to the position of the focused spot on 
the retina for any given eye position just as if an 

15 image were scanned at an infinite distance away from 
the viewer. The intensity of the light is modulated 
by the video signal in order to create an image of 
desired contrast. Therefore, when the user's eye 
moves, the user will perceive a stationary image 

20 while he looks at different parts of the scene. The 
lateral extent of the image is proportional to the 
angle of the scan. Anamorphic optics are used as 
necessary to align the scanned photons and to scale 
the perceived image. By forming a reduced image of 

2 5 the scanner aperture, a proportionately larger 

scanning angle is yielded. Other than this, the 
size of the scanner image is irrelevant as long as 
the light enters the eye. 

More particularly, as shown in Fig. 2, light or 

3 0 photons from a photon generator 12 is projected 

through a cylindrical lens 3 0 and a spherical lens 
32 to an acousto-optical deflector 34 that scans the 
photons in a first or horizontal direction. The 
cylindrical lens spreads the light beam from the 
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photon generator 12 horizontally so that it fills 
the aperture of the acousto-optical deflector 34. 
The spherical lens 32 horizontally collimates the 
light which impinges onto the acousto-optical 
5 deflector 34. 

The acousto-optical deflector 34 is responsive 
to a video signal on a line 3 6 that is applied as a 
drive signal to a transducer of the acousto-optic 
deflector 34 to modulate the intensity of the 

10 photons or light from the photon generator 12 and to 
scan the modulated light from the photon generator 
12 in a first direction or horizontally. The video 
signal on line 3 6 is provided by a video drive 
system generally designated 38 that includes a video 

15 controller 42. The video controller 42 may include 
a video generator such as a frame buffer 4 0 that 
provides video signals on a line 56 and respective 
horizontal sync and vertical sync signals. The 
video controller 42 may also include a 

2 0 microprocessor that operates in accordance with 
software stored in a ROM 4 6 or the like and utilizes 
a RAM 48 for scratch pad memory. The horizontal 
sync signal from the video generator 4 0 is converted 
to a ramp wave form by a ramp generator 50 , the 

2 5 horizontal sync ramp waveform is applied to a 
voltage controlled oscillator 52 that provides a 
signal in response to the ramp input having a 
freguency that varies such that it chirps. The 
output from the voltage controlled oscillator 52 is 

30 applied to an amplifier 54 the gain of which is 
varied by the video data signal 56 output from the 
video generator 4 0 so that the video signal 36 
output from the amplifier 54 has an amplitude that 
varies in accordance with the video information on 
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line 56 and that has a frequency that varies in a 
chirped manner. The video signal on line 3 6 is 
applied to a drive transducer of the acousto-optical 
deflector 34. Varying the amplitude of the drive 
5 signal on line 36 with the video information causes 
the acousto-optical deflector 34 to modulate the 
intensity of the light from the photon generator 12 
with the video information. Varying the frequency 
of the drive signal on line 3 6 in a chirped manner 
10 causes the acousto-optical deflector to vary the 
angle at which the light is deflected thereby so as 
to scan the light in a first or horizontal 
direction. 

A spherical lens pair 64 and 68 images the 
15 horizontally scanned light or photons onto a 
vertical scanner 62 wherein a cylindrical lens 68 
spreads the light vertically to fill the aperture of 
the vertical scanner 62. The vertical scanner 62 
may for example be a galvanometer. The vertical 

2 0 sync signal output from the video generator 4 0 is 

converted to a ramp waveform by a ramp generator 58 
and amplified by an amplifier 60 to drive the 
vertical scanner 62. The speed of scanning of the 
vertical scanner 62 is slower than the scanning of 
25 the horizontal scanner 34 so that the output of the 
vertical scanner 62 is a raster of photons. This 
raster of photons is projected directly onto the eye 
2 0 of the user by projection optics taking the form 
of a toroidal or spherical optical element 72 such 

3 0 as a refractive lens, mirror, holographic element, 

etc. 

The toroidal or spherical optical element 72 
provides the final imaging and reduction of the 
scanned photons. More particularly, the toroidal or 
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spherical optical element relays the scanned photons 
so that they are coincident near the entrance pupil 
26 of the eye 20. Because a reduced image of the 
scanner aperture is formed, the deflection angles 
5 are multiplied in accordance with the Lagrange 
invariant wherein the field of view and image size 
are inversely proportional. As the size of the 
scanned photons , i.e. the exit aperture of the 
virtual retinal display are reduced, the field of 
10 view of the image perceived by the eye increases. 

The optical element 72 can be an occluding 
element that does not transmit light from outside of 
the display system. Alternatively the optical 
element 72 can be made light transmissive to allow 
15 the user to view the real world through the element 
72 wherein the user perceives the scanned virtual 
image generated by the display 10 superimposed on 
the real world. Further, the optical element 72 can 
be made variably transmissive to maintain the 
20 contrast between the outside world and the displayed 
virtual image. A passively variable light 

transmissive element 72 may be formed by sandwiching 
therein a photochromic material that is sensitive to 
light to change the light transmissiveness of the 
25 element as a function of the ambient light. An 
actively variable light transmissive element 72 may 
include a liquid crystal material. A photosensor 
can be used with such an element to detect the 
amount of ambient light wherein a bias voltage 
30 across the liquid crystal material is varied in 
accordance with the detected light to actively vary 
the light transmissiveness of the element 72. 

The system described thus far with respect to 
Fig . 2 is monocular . In order to provide a 
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stereoscopic system a second virtual retinal display 
10' may be utilized in parallel with the first 
retinal display 10, the second virtual retinal 
display 10* projecting scanned photons modulated 
5 with the appropriate video information directly on 
the second eye 20' of the user- This provides a 
medium for binocular depth information so that 
displayed objects appear at different depths* Each 
pixel of the object, however, appears at the same 

10 distance from the user which can create a possible 
conflict between the stereoscopic cue and the 
monocular cue where the stereoscopic cue deals with 
the positioning of the object with respect to each 
eye and the monocular cue deals with the focus of 

15 the light of the object being imaged on the retina. 

More particularly, in prior virtual image display 
systems, each monocular image plane was typically 
focused at optical infinity causing each of the 
pixels within the virtual image to appear at one 

2 0 distance. However, the combination of two prior 

monocular systems to form the binocular view created 
a possible conflict between the distance cues and 
the focus or accommodation cue. 

The virtual retinal display of the present 
25 invention overcomes this problem by utilizing an 
accommodation cue 70 either in the monocular display 
system 10 or in the binocular display system formed 
of displays 10 and 10'. The accommodation cue 70 is 
a focusing or depth cue that is controlled to vary 

3 0 the focus or convergence or divergence of the 

scanned photons rapidly to control the depth 
perceived for each picture element of the virtual 
image. Therefore in accordance with the present 
invention true depth perception is obtained by 
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modulating each pixel for depth individually such as 
by controlling the focus, i.e. the convergence or 
divergence, of the individual pixel. The 
accommodation cue 7 0 includes a reflective surface 
5 that changes shape rapidly. For example, a 

miniature mirror having a deformable membrane whose 
shape is altered as the membrane is charged and 
discharged may be used to form the accommodation 
cue. The deformation of the membrane is thus varied 

10 by an electrical drive signal to control the 
convergence or divergence of each pixel for depth. 
The drive of the accommodation cue 70 is provided by 
the video controller 4 2 which may, for example, 
store a Z axis video information buffer in the 

15 memory 48 or in the video generator 4 0 in addition 
to the two dimensional video information in a 
typical frame buffer. 

A further embodiment of the virtual retinal 
display 10 of the present invention is depicted in 

2 0 Fig. 3 for scanning colored photons directly onto 
the retina of a user's eye. As shown in Fig. 3, the 
photon generator 12 includes colored lasers or LEDs 
such as a red photon generator 80, a green photon 
generator 82 and a blue photon generator 84 . If a 

2 5 blue photon generator is unavailable, a yellow 
photon generator may be utilized. The colored 
photons from the generators 80, 82 and 84 are 
modulated with respective RGB video information from 
the video generator 4 0 and then combined by a beam 

30 combiner/dispersion precompensator 86. The output 
of the beam combiner /dispersion precompensator 8 6 is 
projected onto the horizontal scanner 34 by the 
cylindrical lens 30 and the spherical lens 32. It 
is noted tht the horizontal scanner may be other 
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than the acousto-optic scanner shown in Fig. 2. For 
example, a resonant mechanical scanner or various 
types of microscanners as discussed below may be 
used for the horizontal scanner. The horizontally 
5 scanned color modulated photons output from the 
scanner 3 4 are projected onto a dispersion 
compensator 88 the output of which is projected onto 
a prism before being projected onto the vertical 
scanner 62 by the spherical lens pair 64 and 68. 
10 The colored photon raster as scanned from the 

output of the vertical scanner 62 is projected by a 
spherical lens 92 onto an offset mirror 96 which is 
moved by the eye tracker 106 so as to position the 
raster of photons directly onto the entrance pupil 
15 2 6 of the eye 2 0 as the pupil moves. In one 
embodiment, a beam splitter 100 directs an image 
reflected off of the cornea of the eye 2 0 to a lens 
102 and a position sensing diode 104 that is coupled 
to the eye tracker 106 to detect the position of the 
20 pupil 26. In response to the detected position of 
the pupil, the eye tracker correctly positions the 
offset mirror (s) 96 so that the exit pupil or 
aperture of the virtual retinal display is 
approximately aligned with the entrance pupil of the 
25 eye and/or to adjust the scan angle to reflect 
changed video information as described below. 

The instantaneous position of the pupil 26 as 
determined by the eye tracker 10 6 is also 
communicated to the video controller 42 so that the 
3 0 microprocessor 44 can direct video information to 
modulate the colored light where the video 
information reflects a change in the direction of 
the user's view. More particularly, the detected 
pupil position is used by the microprocessor 44 to 
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position a "visible window" on the video information 
stored in the frame buffer 40, The frame buffer 4 0 
may for example store video information representing 
a panoramic view and the position of the visible 
5 window determines which part of the view the user is 
to perceive, the video information falling within 
the visible window being used to modulate the light 
from the photon generator 12 . 

It is noted that, because the acousto-optical 

10 deflector 34 diffracts red light more than green 
light and diffracts green light more than blue 
light, this variation in the diffraction must be 
compensated for. In accordance with the present 
invention, this variation in diffraction may be 

15 compensated for by appropriately delaying via delays 
108, 110 and 112 the RGB video signals that are 
coupled to the respective red, green and blue photon 
generators 80, 82 and 84 to modulate the red, green 
and blue photons with the appropriate red, green and 

2 0 blue video information. 

In another embodiment of the virtual retinal 
display of the present invention as shown in Fig. 4, 
composite video or RGB video signals are received by 
a digital video scan converter 12 0 and separated 

2 5 into multiple compartments that represent sectors or 

regions of an image to be scanned. Multiple video 
drive signals output from the video amplifiers 12 4 
representing each sector are used to modulate the 
light from the photon generator 12 in parallel. The 

3 0 photon generator may consist of either arrays of 

lasing diodes or arrays of high luminance light 
emitting diodes. Multiple beams of red, green and 
yellow or blue light are modulated with the video 
signals in parallel for each of the divided sectors 
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or regions and then relayed directly or by 
monofilament optical fibers 131 to a microscanner 
16. The microscanner 16 essentially performs two 
functions. First, the microscanner scans the 
5 multiple color beams associated with each sector or 
region in two axes to create a raster of light on 
the retina and not an aerial image, there being no 
image plane between the photon generator 12 and the 
eye 20. Second, the microscanner 16 functions to 
10 position the scanned light relative to the 
instantaneous entrance pupil 2 6 of the eye as sensed 
by the eye tracker 24. 

More particularly, the scanner 16 includes a 
first microscanner 13 2 that is responsive to an X 
15 axis deflection signal output from a deflection 
amplifier 13 6 to scan the color beams in a 
horizontal direction where the amplifier 13 6 is 
driven by the horizontal sync signal from a scan 
generator 122. A second microscanner 134 is 
2 0 responsive to a Y deflection signal from the 
deflection amplifiers 136 as driven by the vertical 
sync or deflection drive from the scan generator 122 
to scan the horizontally scanned color photons in 
the vertical direction. A scan collimation lens 140 
25 receives a two dimensionally modulated light field 
that is projected onto a tri-color combiner 142. 
The combiner 14 2 in turn projects the scanned light 
onto a Maxwellian-view optical system 148. The 
optical system 148 projects the scanned colored 
30 photons onto a raster position deflector which may 
include two axis galvo mirrors that in turn project 
the scanned light onto a toroidal optical element 
such as a combiner 152 having a trichoric coating, 
the toroidal combiner 152 projecting the scanned 
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color photons directly onto the eye 20, 

For eye tracking, the eye tracker 2 4 includes 
an infrared light source which illuminates the 
surface of the eye with low intensity infrared light 
either directly or indirectly as shown. The surface 
of the eye is viewed through the raster position 
deflector 150 via the combiner 142, a lens 140 and a 
charge coupled device, CCD, array 14 6. The signals 
from the CCD sensor 14 6 are processed by a pupil 
position processor 154 to generate null signals, AH 
and AV, that are coupled to respective color 
deflection amplifiers 158 and to the raster 
positioning mirrors 150 so as to cause the scanned 
photons to follow the pupil of the user's eye 20. 

An example of a light emitting diode array 
suitable for use in the present invention is 
illustrated in Fig. 5. If an X-Y visual field is 
considered to be composed of an array of 2,000 x 
2,000 resolvable spots or pixels, the spots must be 
refreshed 50 times per second so as to have an 
information bandwidth of approximately 200MHz. High 
brightness LEDs typically have a power bandwidth 
curve that starts to roll off above 2MHz . This 
result is essentially an R-C product limitation 
related to the diffusion capacitance of a heavily 
forward-biased p-n junction. In order to meet the 
bandwidth requirements of the system, a linear array 
of 50 to 100 LED pixels per color are utilized. 
Using a red, green and blue LED scheme would require 
50-100 LEDs of each of these three colors. As shown 
in Fig. 5, an array 200 includes LED chips 201, 202, 
2 03 - N wherein each LED chip includes an LED active 
area 205. The LED active area may include a GaAsP 
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alloys and a Si 3 N 4 dielectric overlayer. 

A laser phased array as illustrated in Fig. 6 
functions to perform photon generation, video 
modulation and scanning in at least one direction. 
5 The laser phased array includes a thin film wave 
guide 210, phase modulator electrodes 212, a cleaned 
coupled cavity 214 and laser cavities 216, the array 
emitting a coherent beam of about lOmW power. 
When two closely spaced lasers are fabricated 

10 in the same chip of material, their optical fields 
become coupled so that the processes of optical 
emission in the two devices are correlated and 
coherent. The result is a well defined phase front 
emitted from the laser pair. In the laser phased 

15 array 220 having a number of laser cavities 216, the 
optical beam is phase coherent if the lasers are 
spaced within 10 microns of each other. This 
resolution can be achieved by photolithographic 
techniques. The electro-optic modulator works by 

2 0 modifying the index of refraction of the wave guide 

medium 210 through which the optical beam must 
travel before being launched into free space. By 
separating the electrical contacts 212 for each 
modulator, the relative phase of each individual 
25 laser in the array can be modified by the modulator. 

For an appropriate series of modulation voltages, 
the phase front of the laser array coupled beam can 
be modified so that the emitted beam is launched at 
an angle to the normal exit direction. With the 

3 0 appropriate series of modulation voltages the laser 

beam can be scanned in a given direction. It is 
possible to construct a two axis laser phased array 
so that an additional scanner is not needed to scan 
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the laser in a perpendicular direction. 

An example of a microscanner 13 2, 134 for 
scanning photons is illustrated in Fig. 7. The 
microscanner includes an actuator 23 0. The actuator 
230 is a piezoelectric bimorph cantilever that is 
capable of three dimensional motion in response to 
an electrical drive signal. By controlling the 
deflection of the cantilevered actuator with the 
appropriate drive signals, the actuator 230 deflects 
the photons incident thereto to scan the photons. 

Another example of a microscanner that can be 
made extremely small is shown in Fig. 8 , the 
microscanner having a curved reflective surface that 
translates to scan light impinging thereon in one 
15 direction. More particularly, the microscanner 240 
includes a base or actuator 242 formed of a 
piezoelectric material with a substrate 244 formed 
on the actuator 242 wherein the substrate 244 has a 
curved reflective surface 24 6. m response to a 
2 0 varying drive signal the piezoelectric actuator and 
the substrate 244 translate in the direction of the 
arrows 248 so as to scan the light impinging on the 
surface 246 of the substrate in a first direction 
generally perpendicular to the direction 248 of 
25 translation. a second microscanner 250 scans the 
light impinging thereon in a second direction 
perpendicular to the first direction so as to scan a 
raster image directly onto the retina of a user's 
eye . 

30 Many modifications and variations of the 

present invention are possible in light of the above 
teachings. Thus, it is to be understood that, 
within the scope of the appended claims, the 
invention may be practiced otherwise than 
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described hereinabove • 
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CLAIMS 

1. A virtual image display system comprising: 
a source of photons modulated with video 

information, said photons being scanned directly 
onto the retina of the user's eye to produce a 
5 virtual image perceived by the user without an 
aerial image outside of the user's eye perceivable 
by the user* 

2. A virtual image display system as recited 
in claim 1 wherein said display system is mounted on 
a user's head. 

3. A virtual image display system as recited 
in claim 1 wherein said source of photons generates 
coherent 1 ight . 

4. A virtual image display system as recited 
in claim 1 wherein said source of photons includes 
colored light generators. 

5. A virtual image display system as recited 
in claim 1 wherein said source of photons includes 
an array of non-coherent light emitting elements. 

6. A virtual image display system as recited 
in claim 1 wherein said source of photons includes a 
laser phased array . 
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7* A virtual image display system as recited 
in claim 1 wherein said video information represents 
a plurality of picture elements of a video image and 
said system includes means for varying the focus of 
5 said scanned photons to control the depth perceived 
for each picture element of said video image. 

8. A virtual image display system as recited 
in claim 1 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a photon deflector for moving said 

5 scanned light to approximately coincide with said 
entrance pupil. 

9. A virtual image display system as recited 
in claim 8 including a source of video information, 
said video information source being responsive -to 
movement of said pupil position to vary the video 

5 information with which said photons are modulated. 

10. A virtual image display system as recited 
in claim 1 wherein said video information includes a 
plurality of video signals representing a plurality 
of regions of a video image, said photons being 

5 modulated by said plurality of video signals in 
parallel and scanned in parallel. 
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11 . A virtual image display system to be 
mounted on the head of a user comprising: 

a source of photons modulated with video 
information, said photons being scanned in a first 
direction and in a second direction generally 
perpendicular to said first direction directly onto 
the retina of the user ' s eye to produce a virtual 
image perceived by the user without an aerial image 
outside of the user's eye perceivable by the user. 

12. A virtual image display system as recited 
in claim 11 wherein said source of photons generates 
coherent light. 

13. A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
colored light generators. 

14. A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
an array of non-coherent light emitting elements. 

15. A virtual image display system as recited 
in claim 11 wherein said source of photons includes 
a laser phased array. 

16. A virtual image display system as recited 
in claim 11 wherein said video information 
represents a plurality of picture elements of a 
video image and said system includes means for 
varying the focus of said scanned photons to control 
the depth perceived for each picture element of said 
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video image. 

17* A virtual image display system as recited 
in claim 11 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a photon deflector for moving said 
5 scanned light to approximately coincide with said 
entrance pupil, 

18. A virtual image display system as recited 
in claim 17 including a source of video information, 
said video information source being responsive to 
movement of said pupil position to vary the video 

5 information with which said photons are modulated. 

19. A virtual image display system as recited 
in claim 11 wherein said video information includes 
a plurality of video signals representing a 
plurality of regions of a video image, said photons 

5 being modulated by said plurality of video signals 
in parallel and scanned in parallel. 

20. A virtual image display system to be 
mounted on the head of a user comprising: 

a source of photons modulated with video 
information, said photons being scanned in a first 
5 direction and in a second direction generally 
perpendicular to said first direction; and 

optics to project said scanned photons directly 
onto the retina of the user 1 s eye to produce a 
virtual image perceived by the user without an 
10 aerial image outside of the user's eye perceivable 
by the user. 
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21. A virtual image display system as recited 
in claim 20 wherein said optics include a toroidal 
optical element. 

22. A virtual image display system as recited 
in claim 2 0 wherein said optics include a light 
transmissive optical element, said virtual retinal 
display forming a see-through display wherein said 

5 scanned photons are perceived by the user as being 
superimposed upon the real world. 

23. A virtual image display system as recited 
in claim 2 2 wherein said optical element has 
variable light transmissivity . 

24. A virtual image display system for 
projecting an image directly onto an eye of a user 
comprising: 

a source of light; 
5 means for modulating light form said light 

source with a video signal; 

means for scanning said modulated light; and 
optics for projecting said scanned light 
directly onto an eye of the user to produce a 
10 virtual image perceived by the user without an 
aerial image outside of the user's eye perceivable 
by the user. 

25. A virtual image display system as recited 
in claim 24 wherein said display system is mounted 
on a user's head. 
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26. A virtual image display system as recited 
in claim 24 wherein said source of light generates 
coherent light. 

27. A virtual image display system as recited 
in claim 24 wherein said source of light includes 
colored light generators. 

28. A virtual image display system as recited 
in claim 24 wherein said source of light includes an 
array of non-coherent light emitting elements. 

29. A virtual image display system as recited 
in claim 24 wherein said source of light includes a 
laser phased array. 

30. A virtual image display system as recited 
in claim 24 wherein said video information 
represents a plurality of picture elements of a 
video image and said system includes means for 

5 varying the focus of said scanned light to control 
the depth perceived for each picture element of said 
video image. 

31. A virtual image display system as recited 
in claim 24 including an eye position detector for 
detecting the position of an entrance pupil of the 
user's eye and a light deflector for moving said 

5 scanned light to approximately coincide with said 
entrance pupil. 
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32. A virtual image display system as recited 
in claim 31 including a source of video information, 
said video information source being responsive to 
movement of said pupil position to vary the video 

5 information with which said light is modulated. 

33. A virtual image display system as recited 
in claim 24 wherein said video information includes 
a plurality of video signals representing a 
plurality of regions of a video image, said light 

5 being modulated by said plurality of video signals 
in parallel and scanned in parallel. 

34. A virtual image display system as recited 
in claim 24 wherein said optics include a toroidal 
mirror. 

35. A virtual image display system as recited 
in claim 24 wherein said scanning means and optics 
are mounted on a head of a user, said source of 
light and modulating means are remote from said 

5 scanning means and including means for coupling said 
modulated light to said scanning means. 

36. A virtual image display system for 
projecting an image directly onto an eye of a user 
comprising: 

a source of light; 
5 a source of video information, said video 

information modulating light from said light source; 
means for scanning said video modulated light; 

and 

optics for projecting said scanned light 
10 directly onto a user's eye to produce a virtual 
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image perceived by the user without an aerial image 
outside of the user's eye perceivable by the user. 

37. A virtual image display system for 
projecting an image directly onto an eye of a user 
as recited in claim 3 6 wherein said light source is 
a source of coherent light. 

38 A virtual image display system for 
projecting an image directly onto an eye of a user 
as recited in claim 3 6 wherein said source of light 
is a source of non-coherent light. 

39. A display apparatus comprising: 

a light source for emitting a laser beam; 

an optical modulating system for optically 
modulating the laser beam from the light source in 
5 response to a video signal; 

a scanning system for horizontally and 
vertically scanning the modulated laser beam; and 

a projecting optical system for projecting the 
scanned laser beam from the scanned system to form 
10 an image of the laser beam on the retina of an eye 
of an observer of the display apparatus. 

40. A direct viewing picture image display 
apparatus comprising : 

a light source for emitting a laser beam; 

video signal supply means for supplying a video 
5 signal; 

modulating means for optically modulating the 
laser beam emitted from the light source in response 
to the video signal supplied from the video signal 
supplying means; 
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10 scanning signal supplying means for supplying a 

scanning signal; 

scanning means for horizontally and vertically 
scanning the modulated laser beam from the scanning 
signal in response to the scanning signal supplied 
15 from the scanning signal supplying means; and 

a projecting optical system for projecting the 
scanned laser beam received from the scanning means 
to form an image of the laser beam on the retina of 
an eye of an observer of the direct viewing picture 
20 image display apparatus. 

41. A direct viewing picture image display 
apparatus comprising : 

a light source for emitting a laser beam; 
video signal supplying means for supplying a 
5 video signal; 

modulating means for optically modulating the 
laser beam emitted from the light source in response 
to the video signal supplied from the video signal 
supplying means; 
10 depth signal supplying means for supplying 

depth information; 

a variable focus optical system for providing 
the depth information supplied from the depth signal 
supplying means to the modulated laser beam received 
15 from the modulating means; 

scanning signal supplying means for supplying a 
scanning signal ; 

scanning means for scanning the laser beam 
received from the variable focus optical system in 
20 response to the scanning signal supplied from the 
scanning signal supplying means; and 

a projecting optical system for projecting the 
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scanned laser beam received from the scanning means 
to form an image of the laser beam on the retina of 
2 5 an eye of an observer of the direct viewing picture 
image display apparatus. 

42. The direct viewing picture image display 
apparatus as recited in claim 3 9 further including a 
variable focus optical system. 

43. An apparatus for directly displaying an 
image directly on the retina of an eye comprising: 

a plurality of colored lights; 

means for modulating each of said colored 
lights with video information; 

means for combining said modulated colored 
lights into a single beam; 

means for horizontally and vertically scanning 
said single beam of light; and 

optics for projecting said scanned light onto 
an eye of a user. 



5 



10 



9409472A1JA> 




BNSDOCID: <WO 9409472A1 JA> 



RECTIFIED SHEET {RULE 91) 




ftttnflED SHEET (RULE 91) 

BNSDOCID: <WO 94094 72A1_IA> 




RECTIFIED SHEET (RULE 91) 



BNSDOCID: <WO 94094 72A1 _IA> 



WO 94/09472 



4/6 



PCI7US93/09467 




WO 94/09472 5/6 ^ PCI7US93/09467 





BNSDOCID: <WO 9409472A1 JA> 



WO 94/09472 



6/6 



PCT/US93/09467 




BNSOOCtD: <WO 94094 72A1 _l A> 



INTEMSWiIONAL SEARCH REPORT 



Inrurnational application No. 
PCT/US93/09467 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC(5) :G09G 3/00. 3/20,3/22 

US CL : 345/8; 359/630; 
According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 345/7, 8, 9; 359/630, 631; 340/980 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data ba»e consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


A 


US, A, 4,361,384 (Bosserman) 30 NOVEMBER 1982, coL 4 


1-43 




lines 42-44. 




A 


Information Display, published in September 1987, article by 


1-43 




Jim E. Wurtz, "Applications of miniature CRT displays", 






pages 16-18, note especially the first paragraph on page 17 






after the title "Helment-mounted Displays". 





| Further documents are listed in the continuation of Box C. | | See patent family annex. 



1 Special categories of cited documeat*: 

A* document defining the general state of the art which is not considered 

to be put of particular relevance 

E* earlier documeat published oo or after the rateraaoooal filing date 

L" document which may throw doubta on priority cloim(i) or which » 

cited to establish the publication dote of another citation or other 
special reason (as specified) 

O" document referring to an oral disclosure, use, exhibition or other 

mean* 

P* document published prior to the mtemahonal filing date but later than 



*T* later document published after the international filing date or priority 

date and not in conflict with tbe application but cited to understand the 
principle or theory underlying the mvention 

"X" document of particular relevance; tbe claimed invention cannot be 

considered novel or cannot be considered to involve sn inventive step 
when the document m taken alone 

• y* document of particular relevance; the churned invert boo cannot be 

considered to involve an inventive step when tbe document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

'&.* document member of the same patent family 



Date of the actual completion of the international search 
01 December 1993 


Date of mailing of the international search report 

, FEB 23 lS94 / 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. NOT APPLICABLE 


Authorj»d 

lEtjFERY A. BRIER 
Telephone No. (703) 305-4700 



Form PCT/1SA/210 (second shcct)(JuIy 1992)* 



BNSOOCID: <WO 9409472A1 J A> 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 



□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: __ ■ 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




BLURRED OR ILLEGIBLE TEXT OR DRAWING 



This Page Blank (uspto) 



